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ABSTRACT 

This report presents a reviev of follov^up studies of 
foraer occupational-technical students at coanunity colleges 
throughout the nation. The purpose vas to evaluate the effectiveness 
of follov-up studies in measuring attainment of educational goals 
stated in coBBonity college catalogs and literature. This evaluation 
was Bade by studying the research emphases, activations, and 
procedures for the studies. In winter 1973, a 12-itea questionnaire 
was sent to all 720 public coaprehensive two-year colleges which 
began offering occupational-technical prograas prior to Septeaber 
1970; 520 (72.5 percent) responded. This questionnaire requested 
inforaation on research activations and frequency, and asked for 
copies of recent studies. The content of the 65 aost sophisticated 
follow-up studies subaitted was analyzed using a three-part 66-itea 
coding instruaent. A 44-itea goals inventory for 

occupational-technical education was prepared in order to aeasure the 
educational goal eaphases of the studies. Findings are reported and 
recoaaendations for iaproving follow-up studies are aade. The 
appendix includes a list of the 42 coaaunity college catalogs and a 
bibliography of the literature consulted in deternining educational 
goals, a list of the 85 colleges subaitting sophisticated follow-up 
studies, the questionnaire and accoapanying letters, and the content 
coding fora and procedures for its use. (Author/DC) 
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INTRODUCTION 



Purpose of the Investigation 

This report presents a review of follow-up studies of former 
occupational-technical students at community colleges throughout the 
nation. The purpose was to evaluate the effectiveness of follow-up 
studies in measuring attainment of educational goals, as stated in 
community college catalogues and literature. This evaluation was 
made by studying research emphases, motivations, procedures, design 
influences, and report formats of community college research projects. 
Data were obtained from questionnaires returned by more than 500 
community colleges and from a content analysis of 85 accompanying 
follow-up study reports. Research emphases In follow-up reports were 
compared with a 44-item goals inventory prepared by the project staff 
and consultants. 

The findings of this study are presented in terms of 18 research 
questions about follow-up studies. Twelve recommendations for improving 
follow-up studies of former occupational -technical students were derived 
from the findings. 

This research was supported by federal funds made available through 
the Virginia Department of Education, Vocational Education Division. 
A proposed review of follow-up study findings and a comparison of such 
findings with those of Virginia community college research are not 
Included but will be the subject of future projects. 



Importance of Fo I low-Up Research 

Evaluation of occupational -technical education at the community 
college can take many forms. One method is to administer ability tests 
before and after courses of Instruction. Another Is to examine 
conditions of resources, such as quality of instruction and facilities. 
A third method, and the subject of the present report, is to examine 
post-college experiences and perceptions of former students in order 
to determine whether educational goals have been achieved. This 
procedure involves evaluation through follow-up research. 

Follow-up studies are important In establishing public c rfidence 
and support among citizens, legislators and executives In government. 
Hamlin (1967) believes that "Citizen evaluation of public occupational 
education is probably the most Important factor 'affecti ng it. On the 
basis of their evaluations, citizens assign responsibilities for 
occupational education and provide or withhold funds, personnel, and 
facilities (p. III)." Public support Is a major focus of accountability. 
Tyler (1969) writes that "Only as we can describe more accurately the 
results we are obtaining from the curriculum are we In a position to 
get the most intelligent support for the educational program . . . 
(p. 125)." Rouche and Boggs (1968) refer more directly to financial 
Implications of accountability when they state that "As a consequence 



of the increased need for funds, efficiency-minded legislators, parents, 
boards of trustees, and the public are asking whether institutions are 
getting the maximum value from each dollar expended (p, I)." O'Connor 
(1965) proclaims that "The existency of junior colleges depends on their 
ability to convince the people in their communities that they are dynamic, 
educational institutions capable of serving students well (p. 52) 



Need for More Information on Fol low-Up Studies 

Follow-up study reports on former occupational -technical students 
are only rarely published or widely circulated, and reviews of sucn 
studies have been inadequate. The isolation of follow-up research reduces 
its usefulness in at least two ways. First, progress In research desigrr 
is impaired by a lack of dissemination of research ideas among community 
col leges. Second, the absence of shared research emphases and procedures 
reduces college-to-college comparability of foi low-up research data. The 
present report provides a review of follow-up s'*'udles of former occupational- 
technical students which purports to communicate useful research Ideas, 
facilitate the comparability of research data, and promote the relevance 
of future follow-up research to stated goals. 

Writing on the subject of community college follow-up research, 
O'Connor (1965) observed that follow-up studies are particularly valuable 
in measuring the benefits of career education. He went so far as to say 
that "hto better method exists for determining how well the technlcal- 
occupational objectives of the college are being carried out (p. 22)*" 

Despite the value of follow-up research there seems to be no consensus 
as to how such studies should be conducted. Garland and Carmody (1970) 
found little agreement among institutions about adequate Indicators of 
success In occupational-technical programs. The only widely accepted 
Indicator was employment rates of former students. Huff (1971) also 
argues for greater standardization of measurement in evaluation research, 
pointing out that inter-Institutional exchange of data and study of a 
longitudinal nature require standardization. Describing their Investigation 
of institutional research in two-year colleges, Rouche and Boggs (1968) 
conclude that "Many two-year colleges are engaging In some research activity, 
but that dissemination of findings Is typically limited to the institution 
Involved (p. 52)." 



Review of Related Literature 



Reviews of Fol low-Up Research 

Gartland and Carmody (1970, pp. 15-23) surveyed 689 two-year colleges 
which offered both transfer and occupational-technical programs. Their 
purpose was to learn more about institutional guidance and research programs. 
They also sent questionnaires to 351 vocational-technical schools and 
requested copies of follow-up studies from all Institutions. 
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Of 518 two-year colleges responding, 18 percent reported that 
they had never conducted follow-jp studies and 28 percent stated that 
they rarely conducted follow-up studies. However, more than one-half 
(55%) reported that follow-up studies of former occupational -technical 
students were conducted on a regular basis. Institutional spokesmen 
for 99 percent of the two-year colleges reported that they believed 
follow-up studies to be useful. 

Based on a review of 45 follow-up study reports which were 
received from both vocational-technical schools and two-year colleges, 
Gartlani and Carmody (1970, pp. -23) observed that very few of the 
studies included students who hao not completed programs. Only one 
study included employer evaluations of student preparation. The 
authors found the range of student response rates to be between 30 
and 85 percent, with a mean response rate of 60 percent. 

Regarding the common failure to obtain employer evaluations, the 
authors stated that "The apparent lack of interest in this area is 
surprising since it is likely that detailed evaluations of program 
graduates by employers would provide institutions with valuable 
information concerning the effectiveness and relevance of their 
occupational curricula (p. 23)." 

Deem's (1969, pp. 18-38) dissertation was intended to compare 
existing practices in follow-up research at public two-year colleges 
with the practices desired by administrators at the colleges. One of 
his secondary goals was to compare follow-up research conducted in 
Illinois with that conducted elsewhere. 

The method used by Deem was to send questionnaires to the 
institutions and their instructors and administrators. He did not 
review follow-up studies firsthand. His sample of institutions was 
intentionally non-representative of American public two-year colleges 
and the number of respondents was rather low. 

In his introduction. Little (1970) stated his intention to 
review follow-up research on the secondary, post-secondary and adult 
levels. However, nearly all of the research reports and findings 
which were presented pertained only to secondary education. Little's 
review, therefore, does not meet the need for a state of the art 
review of community college follow-up research regarding fontier 
occupational -technical students. 



Fol low-Up Guides 

Comprehensive guides on planning, implementing and reporting 
follow-up studies of occupational -technical students would facilitate 
uniformity of research design a-^ community colleges. Two guides have 
been found which have some usefulness. 



Hulnker (1970) attempted to design a model follow-up plan for both 
transfer and occupational -technical students who ^ere formerly enrolled 
In community colleges* His dissertation (p. 35) contains severe! helpful 
guidelines on follow-up research which are listed below: 

Provide for democratic Involvement by members of the college 
community. 

Include all categories of the student population. 
Insure utilization of the results of the study. 
Appoint a single, responsible coordinator. 

In a six-page master plan, Hulnker gave advice on what should be 
done In planning, conducting and reporting follow-up studies. However, 
very little Is said about how the recommended procedures are to be 
carried out. 

O'Connor (1965, pp. 9-10) offered Informal advice about follow-up 
research. Some of his most useful Ideas concern the role of follow-up 
studies for different members of the college community. O'Connor points 
out that follow-up studies provide counselors with a factual basis for 
advising students on career choice and course selection. Also, students 
can obtain information on the level of success they may achieve after 
college, what salaries they can earn, what their chances for transfer 
will be, and why they must take certain required courses. Finally, 
Instructors can learn about the adequacy of their teaching and course 
content. 

O'Connor (p. 43) provided a list of relevant data for follow-up 
research, which Included the following: 

Dates of enrollment and major field of study 

Present job and relatedness to training 

Job History since leaving college 

Student evaluations of Instruction and student services 
Additional training after attendance 
Advice to other students 

Each of these Items Is useful, but more details are needed on ways to 
collect and Interpret data and relate them to educational goals. 



Nationwide FoMow-Up Studies 

The existence of comprehensive nationwide follow-up studies could 
solve some of the problems which result from Isolation of follow-up 
research. A nationwide study would receive widespread attention and 
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establish baseline data with which colleges could compare th 'r own 
findings. Unfortunately, a model nationwide follow-up study ^f 
occupational -technical students at community colleges does not exist. 

Somers' (1971) reported on a federally funded national follow-up 
study of former vocat ional-technica I students conducted by the Center 
for Studies in Vocational and Technical Education at the University of 
Wisconsin and by the Bureau for Social Science Research, Inc. Somers' 
questionnaire ?s lengthy (63 questions) but it is a good resource 
document. While the author admitted to several disappointments in the 
study, many of his procedures could be used to advantage by community 
colleges. For example, repeated student contacts and tests of non- 
response bias were used in an effort to maximize the response rate 
and determine the degree to which respondents were representative of 
the study sample. The elimination of non-graduates from the sample is 
unfortunate and reduces the usefulness of Somers' work as a research 
model . 



Educational (Soals Inventories 

The problem of defining the goals of post-secondary education nas 
been confronted many times in research reports, conferences, books, and 
journal articles. Unfortunately, a careful review of the literature 
has revealed no single comprehensive goals inventory appropriate for 
occupational-technical education. However, several of the most valuable 
inventories deserve mention. 

Huff's (1971) inventory of the*benefits of higher education was a 
result of the work of the Planning and Management Systems unit of the 
Western Inter-State Commission on Higher Education, to develop means 
to measure outputs of educational programs. The outcomes and activities 
in the Huff inventory are arranged in four categories: Research Outcomes, 
Publ ic Service Outcomes, Instructional Outcomes, and Instructional 
Environment Variables. The outcomes under the Instructional Outcome 
heading are further categorized as Cognitive, Affective and Tangible. 
Valuable goals can be derived from the outcomes and activities listed in 
these three sections of the Instructional Outcomes category. The Huff 
inventory is a fertile resource for planning an educational goals 
inventory. However, the outcomes were not stated in measurable terms. 

The Education Commission of the States published Objectives for 
Career and Occupational Development (1971) as an activity of its project 
entitled National Assessment of Educational Progress (NAEP). The NAEP 
objectives are arranged in five categories: making career decisions, 
gaining occupational skills, obtaining general education capabilities, 
practicing effective work habits, and developing positive attitudes 
toward work. The NAEP inventory is unique in that it outlines the 
levels of attainment of each of the objectives for individuals at 
age 9, 13, 17 and in early adulthood. 

Harold Starr (1970) prepared a System for State Evaluation of 
Vocat iona I Educat ion with a team of experts from The Center for 
Vocational and Technical Education, Ohio State University, and 
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elsewhere. An appendix (p. 29) contains educational goals and objectives 
for vocational education. Most of the goals are input and procedures 
goals rather than output goals, input goals include enrollment in 
vocational education programs of persons receiving public assistance 
and persons with physical or mental handicaps. Procedural goals include 
long range plans, advisory committees, and simulated worl^ experiences 
in the classroom. The 1969 document was an Interim report which may 
have undergone further refinement. 

Mount San Jacinto (Community) College in Gilman Hot Springs, 
California maintains a list of Measurable Institutional Objectives 
which it modifies from year to year. The 1971-72 list contains objectives 
for general education, transfer education, community services, student 
personnel services, remedial education, athletics, and occupational 
preparation. Most of the Mount San Jacinto objectives include specific 
criteria by which achievement of the objectives can be determined. For 
example, 90 percent of reading class students who complete the semester 
are expected to increase their reading skills by at least one grade 
level. Eighty percent of minority students who are In peer tutoring 
and counseling programs are expected to persist through the semester. 
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RESEARCH METHOD 



The initial activity in the project was to identify research 
questions which should be answered by the investigation. Dnce the 
research questions were slated, data-gathering In*^-^ mmts were 
designed to obtain information useful in answer ,uestlons. 
These quesMons are listed below, with a descr.^ . of the relevant 
data-gather i ng i nst ruments . 

A group of consultants (Appendix G) participated in refining the 
procedures and instruments of the study. Each consultant was chosen 
because of his distinguished contributions to post-secondary occupationa 
technical education and educatiopj=jl research. 



Community College Population 

The study population consisted of all 720 public comprehensive 
two-year colleges in the United States which began offering occupational 
technical programs prior to September, I 970. These colleges could be 
expected to have sufficient numbers of former students to warrant follow 
up reseai ch. 



Data-Gather i ng I nstruments 

Community College Questionnaire 

A twelve-item questionnaire (Appendix B) was designed to solicit 
information about follow-up studies and related research activities 
which could not be obtained directly from a content analysis of follow- 
up reports. The questionnaire was to obtain information useful in 
answering the following research questions: 

1. How many community colleges conduct follow-up studies of their 
former occupationa I -technical students? 

2. What motivates community colleges to conduc*** follow-up studies 
of former occupational-technical students? 

3. Which college personnel are most important In the design of 
follow-up studies of former occupational-technical students and what 
external design sources are used? 

4. If the follow-up study designers have attempted to consult 
follow-up studies conducted by other colleges, are the designers t.^ 
satisfied with the availability of other studies? 

5. In addition to follow-up studies, how many colleges conduct 
exit interviews of departing students, and how many make comparison 
studies regarding non-students? 



iL3 



6. How many community cx5l leges have had their students included 
In follow-up studies designed and conducted by some off-campus agency? 

Several items were included In the questionnaire to confirm whether 
the college met conditions for Inclusion In the study as noted on the 
previous page. A flela test and revision of the questionnaire was 
conducted by submitting it to several college presidents in the Virginia 
Community College System. 



Follow-Up Study Coding Instrument 

A three-part, 66- item coding Instrument (Appendix D) was designed 
to analyze the content of the follow-up study reports which were 
furnished by cooperating Institutions. A panel of consultants 
(Appendix G) reviewed and approved the coding Instrument as being 
suitable for answering the following research questions: 

7. What student populations or samples are studied and how are 
subjects selected? 

8. How many a+tempts are made to obtain student replies, and what 
response rates are obtained with varying numbers of contacts? 

9. What is the general format of reports? For example, what Is 
their length, how many questions are asked, what balance exists between 
narrative and data, and how much unrelated material Is Included? 

10. Are statistical procedures epoloyed, such as tests of non-response 
bias or of the Inferential value of sample-based findings? 

11. What number of community colleges obtain employer evaluations 
of skills and work attitudes of former students? 

12. Are data commonly controlled for pre-enrol Iment experiences of 
former students? For example, do colleges report current salaries of 
former students without Identifying students who had substantial prior 
experience or education In their occupation? 

13. Do colleges conduct longitudinal studies In order to measure 
career advancement or obtain more mature reflections of their former 
students? 

14. Given frequently low student questionnaire response rates, 
are questions being asked which could be eliminated by obtaining the 
data from college records? 

15. How many colleges request baccalaureate and post-bajca I aureate 
transfer Information from former occupational-technical students? 

16. How many colleges solicit educational evaluations and advice 
from former students? 
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17. To what extent are follow-up studies designed to measure 
attainment of educational goals? 

18. Which goals receive the greatest amount of attention by 
college follow-up researchers? 



Educational Goals Inventory 

Research questions 17 and 18 required the development of an 
educational goals inventory for post-secondary occupational-technical 
education. This section describes the development of the goals Inven+ory, 
which appears in the next section of this report. The goals in the 
Educational Goals Inventory for Community College Occupational -Technical 
Education were derived from three types of sources: community college 
catalogues, published works on the goals of education, and ideas of the 
project staff and consultants. 

The catalogues of 42 community colleges (Appendix A) were reviewed 
for statements of educational goals. The colleges were selected by a 
fixed interval procedure from a list arranged alphabetically within 
states. Goals were taken primarily from each cataloguers statement of 
purpose or phi'osophy, from the introduction to sections on career 
programs, and from course descriptions. In order to enlarge the 
inventory, publications on educational goals were reviewed for ideas 
which did not receive attention in the college catalogues. This goal- 
related literature is discussed in the preceding section. 

The educational goals were arranged in six categories for ease in 
presentation. These goals were reviewed by the consultants who found 
them to be satisfactory af+er modification. Examples of information 
which might be gathered by community college personnel to measure 
attainment of goals were also selected. The sample evaluation measures 
are presented as examples and not as a comprehensive plan. 

Procedure 

Project Questionnaire 

In February 1973, the community college questionnaire (Appendix B) 
was sent to the colleges in the study population. A covering letter was 
included which described the general nature of the investigation and 
invited colleges to furnish reports of their follow-up studies of 
occupational-technical students. On March 5, there was a second 
mailing which included a reminder letter (Appendix C) and a duplicate 
of the original letter and questionnaire. Table I shows the status 
of returns at the April 15 cut-off date. Completed questionnaires 
were received from 72.5 percent of the eligible two-year colleges. 



Follow-up S tudy Content Analysis 



Reports from 85 institutions (Appendix F) were Included In the 
content analysis. Each study was coded In accordance with the coding 
Instructions which appear In Appendix E. Not every Item submitted by 
community colleges was considered by the project staff to be appropriate 
for content analysis. Table 2 shows the final disposition of materials 
from the 149 institutions which stafed a willingness to furnish copies 
of follow-up studies. Eighteen colleges failed to send reports, and 18 
subml+ted Vocational Education Completions and Placement Reports (OE 
Form 3139). Such reports are required by the U. S. Office of Education 
and were not considered by the authors to represent college-Initiated 
follow-up research. Therefore, the federal reports were excluded from 
the content analysis of follow-up studies. The 28 other reports not 
accepted for coding consisted of a variety of documents which could not 
be considered follow-up studies. A brief list of examples appears below: 

Student questionnaires not accompanied by reports 

Demographic data and/or attitudes of present (not former) students 

Follow-up reports limited to former baccalaureate-transfer students 

Periodical articles about follow-up studies 

Correspondence regarding follow-up research 

Service area manpower needs studies 

In cases where a college submitted a series of annual reports 
concerning consecutive graduating or departing classes, only the most 
recent study was coded. It was assumed that the most recent study 
reflected the college's most advanced research emphases and procedures. 
In cases where colleges submitted a group of studies concerning the same 
students, the group of studies was coded In the aggregate. For example, 
a college might publish Its follow-up study ir parts or volumes, one 
dealing with attrition, another with post-college employment, and another 
with continuing education. 

After the Initial coding, an interim tabulation of results and 
examples of the coding procedure were reviewed by the consultants. 
Following suggestions by the consultants, each of the 85 reports was 
coded a second time. The purposes of the second coding were to make 
minor changes in procedure, to Insure coding ^accuracy, and to utilize 
accumulated coding expertise. 
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TABLE I 



Questionnaire and Report Response by State 









Val id 


Percent 


UsabI 




Two-year 


El igible 


Quest ionna i res 


Response 


Report 


State 


Col leges 


1 nstitutions 


Retu rned 


Rate 


Provi d 


Alabama 


23 


17 


lO 


59 


0 


Al aska 


8 


7 


4 


57 


0 


Arizona 


13 


12 


7 


58 


1 


Arkansas 


8 


2 


2 


100 


1 


Cal i fornia 


99 


93 


72 


77 


10 


Colorado 


16 


16 


15 


94 


1 


Connecticut 


22 


12 


8 


67 


1 


Delaware 


4 


2 


1 


50 


1 


D. C. 


3 


0 


0 


0 


0 


Florida 


32 


27 


23 


85 


5 


Georgia 


23 


12 


7 


58 


1 


Hawai 1 


6 


6 


5 


83 


2 


Idaho 


4 


2 


2 


100 


0 


1 1 1 1 no i s 


55 


46 


32 


70 


8 


1 ndiana 


4 


1 


0 


0 


0 


Iowa 


26 


16 


13 


81 


4 


Kansas 


25 


19 


18 


95 


0 


Kentucky 


23 


14 


9 


64 


0 


Louisiana 


8 


5 


2 » 


40 


0 


Ma i ne 


6 


2 


2 


100 


0 


Maryland 


21 


15 


9 


60 


5 


Massachusetts 


33 


15 


12 


80 


1 


Michigan 


37 


33 


21 


64 


4 


Mi nnesota 


23 


19 


1 5 


79 


1 


Mississippi 


24 


18 


12 


67 


1 


Mi ssour i 


22 


16 


12 


75 


2 


Montana 


3 


3 


3 


100 


0 


Nebraska 


13 


7 


6 


86 


1 


Nevada 


3 


1 


1 


100 


0 


New Hampshire 


4 


0 


0 


0 


0 


New Jersey 


22 


13 


10 


77 


4 


New Mexico 


9 


5 


2 


40 


0 


New York 


61 


42 


33 


79 


8 


North Carol i na 


66 


29 


22 


76 


1 


North Dakota 


5 


5 


3 


60 


1 



TABLE I— Continued 



State 



Two-year 
Co 1 1 eges 



El igible 
Institutions 



VaTTd 
Questionnaires 
Returned 



Percent Usable 
Response Reports 
Rate Provided 



Ohio 38 

Oklahoma 18 

Oregon 1 6 

Pennsylvania 49 

Rhode Island 3 

South Carol ina 26 

South Dakota 2 

Tennessee | 8 

Texas 59 

Utah 5 

Vermont 5 

Virginia 27 

Washington 27 

West Virginia 7 

Wisconsin 31 
Wyoming . 7 



16 
12 
13 
31 
I 

7 
0 
6 
42 
4 

0 
18 
25 
3 
3 
7 



II 
8 
8 

17 
0 

4 

0 
6 
28 
4 

0 
15 
20 
3 
3 
2 



69 
67 
62 
55 
0 

57 
0 
100 

67 
100 

0 
83 
80 
100 
100 
29 



3 
0 
I 

4 

0 

0 
0 
0 
4 

0 

0 
4 
3 

0 
2 
0 



Totals 



,092 



720 



522 



72.5 



85 



TABLE 2 

Selection of Reports for Coding 



N 

Reports coded g3 
OE Form 3139, Vocational Education 

Completions and Placement Report 18 

Other reports not accepted for coding 28 

Failed to send reports 18 

Total colleges pledging to furnish reports T49 



er|c 



12 18 



EDUCATIONAL GOALS INVENTORY 



One important objective of this investigation was to determine 
whether community college follow-up studies are designed to measure 
attainment of educational goals. It was necessary, therefore, to 
construct an educational goals inventory in order to determine 
whether the data gathered or reported in follow-up studies were 
appropriate for measuring goal attainment. The goals in the inventory 
were derived from community college catalogues, the literature of 
post-secondary education, and ideas of the project staff and consultants. 
Each goal is accompanied by one or more examples of the kind of data 
which can be used to measure goal attainment. The goals appear on 
the following pages, arranged in six categories: 

1. General benefits to the ccTimunlty 

2. Recognition of individual needs 

3. Self-awareness and career choice 

4. SkilU- and attitudes for employment 

5. Rewarding I i ves 

6. Good citizenship 
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FINDINGS 



This section presents the findings of the research project in the 
form of answers to the eighteen research questions listed in Section II, 
Research Method. The first group of findings (research questions I 
through 6) is based on data obtained by questionnaires received from 
522 community colleges (73 percent of the original 720). The second 
group of findings (questions 7 through 18) is based on a content analysis 
of follow-up study reports submitted by 85 co..;nunity colleges. 

The writers believe that the Inferential value of the questionnaire 
data Is not seriously biased by the 27 percent non-response rate. 
However, the findings which are based on the content analysis of 
follow-up reports are not representative of all institutions, because 
less than one-half conducted follow-up studies resulting In written 
reports (Table 3). The 85 reports used in the content analysis 
represent follow-up studies which are more sophisticated than the 
average community college follow-up research. Therefore, the findings 
in this national review may well be biased toward overstating the 
quality of follow-up research by community colleges. 



Research Questions 

1 . How many community colleges conduct follow-up studies of former 
occupational -technical students? 

Eighty-seven percent of the 520 institutions which answered the 
questionnaire item reported that they had conducted follow-up studies 
of former occupational-technical students (Table 3). However, 
approximately one-half of this group reported conducting only informal 
studies which did not result in written reports. Of those surveyed, 
233 community colleges reported conducting formal follow-up research, 
but only 149 institutions agreed to furnish copies of their reports. 
Of those which agreed, 85 sent reports acceptable for content coding, 
46 sent reports not acceptable for coding, and 18 failed to send reports 
(Table 2). Thus, a large percentage of the institutions that reported 
conducting formal follow-up research may not actually have completed 
formal studies. 

2. What motivates colleges to conduct follow-up stud ies of former 
occupational -technical students? 

The questionnaire item on this subject required respondents to 
state whether each of five rroti vat ions was (I) very important, (2) of 
some importance, or (3) of little or no Importance. 

Table 4 presents these motivations In descending order of importance. 
It can be seen that the questionnaire Item did not discriminate 
dramatically between the importance of potential motivations. Curriculum 
evaluation was the motivation most often ranked very important or of some 
Importance. Evaluation of college personnel was the least important 
motivation. 
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TABLE 3 



Status of Follow-up Studies of Former 
Occupational -Technical Students 



N i 



Conducted follow-up studies and 



promised copies of reports to be 






rev i ewed 


149 


29 


Conducted follow-up studies, but 






chose not to send copies 


84 


16 


Conducted informal studies for which 






there are not written reports 


221 


42 


Had not conducted follow-up studies 






of former 0-T students 


66 


13 


Total usable responses 




T5?y 



TABLE 4 

Research Motivations Rated Very Important 
or Of Some Importance 





N 


i 


Curriculum evaluation 


355 


91 


Accountability to the public 


325 


83 


Requests from governing or 






accrediting authorities 


295 


76 


Student services evaluation 


291 


75 


Personnel evaluation 


233 


57 



Total usable respon<;es 390 



3. Which community college personnel are most tmportant In the 
design of follow-up studies of former occupational -technical students 
and what external design sources are used? 

The questionnaire item concerning this research question asked 
respondents to rank design sources as (I) very important, (2) of some 
importance, or (3) of little or no importance (Table 5). 

The least Important factors In shaping the design of community 
college follow-up studies were external consultants and publications 
on methods of conducting follow-up studies. Administrators other than 
research staff members were slightly more important than faculty members 
in follow-up study design. Less than one-half of the respondents 
indicated that research staff members were important in follow-up 
study design. 




TABLE 5 



Design Sources Rated Very Important 
or Of Some Importance 





N 

ii 


<t 


nUMii 111 3 1 1 a isji 3 


346 


88 


Faculty members 


321 


82 


Other follow-up study designs 


247 


63 


Collegers research staff 


179 


46 


Published guides or texts 


66 


17 


Outside Consultants 


53 


14 


Total usable responses 391 







4. If follow-up study designers have attempted to consult follow-up 
studies conducted by other colleges, are the designers sat isfied with the 
availability of other such studies? 

About one-third of the community colleges responding indicated that 
no attempt had been made to refer to the design of other follow-up 
studies in planning their own follow-up research (Table 6)* Of the 
college personnel who attempted to borrow design ideas, less than 
one-third reported satisfaction with the availability of other follow- 
up studies. 

TABLE 6 

College Reports of Satisfaction with 
Availability of Fol low-Up Studies Conducted 
at Other Community Colleges 



Satisfied 

Not Satisfied 

No attempt to consult 

other studies 
Total usable responses 



N 


i 


86 


18 


208 


45 


176 


37 


470 


100 



5, In addition to follow-up studies, how many colleges conduct 
exit interviews of departing students, and what number make comparison 
studies regarding non-students ? 

Seventy percent of responding colleges reported that exit interviews 
of departing students are regularly conducted on their campuses (Table 7). 

o 23 
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TABLE 7 



Community Colleges Which Regularly 
Conduct Exit Interviews 





N 


1 


Conduct exit Interviews 


362 


71 


Do not conduct exit Interviews 


151 


29 


Total usable responses 


5TT 


100 



Studies of non-students are useful for comparing jobs, salaries, and 
citizenship of residents who have not attended the community college with 
those who have. Other information such as non-student educational interests 
and reasons for not enrolling can also be helpful. Of the colleges responding, 
one-third reported that they had conducted studies of non-students in their 
service areas (Table 8). 



TABLE 8 



Community Colleges Conducting Research 
Studies of Non-Student Populations 





N 


i 


Conducted non-student research 


164 


32 


Did not conduct non-student research 


350 


68 


Total usable responses 


514 


100 



6. How many community colleges have had their former occupational- 
technical students included in follow-up studies designed and conducted 
by some off-campus agency? 

Less than one- fourth of the respondents indicated that their former 
students had been included in follow-up studies conducted by state or 
regional follow-up researchers (Table 9).* It appears that most community 
colleges must rely on their own efforts in the follow-up study of former 
occupa t iona I -techn i ca I students . 



*This was not a forced choice questionnaire item. Respondents were 
to supply the name of the researcher and the title of the research report. 
It is possible that some college personnel were not able to recall this 
information. 
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TABLE 9 



Colleges Reporting Students Included 
in Statewide or Regional Fo I low-Up 
Studies Since January, 1967 



Students had been Included In 

off-campus studies 115 22 

No report of off-campus follow-up 405 78 

Tota I CO 1 1 eges respond i ng 520 1 00 



7. What student populations or samples are studied and how are 
subjects selected? 

Of the follow-up reports containing a description of subjects, more 
than three-quarters of the reports concerned only graduates (Table 10). 



TABLE 10 

Selection of Subjects Based 
on Program Completion 









N 


i 


Graduates only 






61 


77 


Both graduates a 


nd 


non-graduates 


18 


23 


Non-graduates on 


ly 




0 


0 


Total reports wi 


th 


i ndlcation 


79 


100 


Reports with no 


1 ndlcation 


6 





Total college populations, rather than sub-populations or samples, 
were included In nearly three-quarters of the 79 studies which adequately 
described subjects (Table II). Such populations normally Include all 
graduates or all former students who were graduated or enrolled during 
a stated period of time. The term sub-population refers to a portion of 
the population chosen by criteria of hours completed, curriculum in which 
enrolled, ov other special designation. By contrast, samples are chosen 
by random or system i zed means In order to Insure that the subjects are 
representative of the larger population from which they are selected. 

Of the 21 institutions which reported selecting samples or sub- 
populations the great majority chose arbitrary sub-populations (Table 12). 
An arbitrary sub-population was most often a designated currlcular group 
or a group of students who accumulated some specific number of credit 
hours. Only two studies used representative sampling procedures. 
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TABLE II 



Choice of Total Population Versus 
Sub-Population or Ref esentatlve Sample 



Tota I Sub-Popu I at Ion 

Population or Sample 

N i Hi 

Graduates only 50 63 II 14 

Graduates and non-graduates 8 10 10 13 

Total reports with Indication 58 73 IT 17 
Grand Total 79 



TABLE 12 

Method of Selecting Subjects 
from a Total Population 



N i 

Arbitrary sub-population 18 90 

System sample I 5 

Random sample I 5 

Total Indicating selection method "25" TOO 
Reports with no method Indicated I 



8. How many attempts are made to obtain student replies and what 
response rates are obtained with varying numbers and types of student 
contact? 

Of the 85 reports reviewed, nine, contained no information about 
response rates or other data from which response rates could be 
calculated. Only 28 reports specified more than one student contcct 
and all others were assumed to have been based on only one contact, 
although such an assumption may not be justified (Table 13). 

Table 14 Is a frequency distribution of response rates for four 
categories of studies: (I) all studies, (2) those which report two 
mcll contacts, (3) those which report three mall contacts, and (4) those 
which report the use of telephone contacts with students regardless of 
mall procedures. The distributions of student response rates are broad 
and blmodal. Although three mail contacts and telephone contacts appear 
to have positive effects on response rates, the n'jmbers Involved are too 
small to justify firm conclusions. 
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TABLE 13 



Reports of Multiple Mall Contacts 
and Telephone Contacts 





N 


i 


Reports specifying two mail contacts 


24 


32 


Reports specifying three mall contacts 


4 


5 


Reports specifying telephone contacts* 


14 


18 


Total reports with response data 76 







*Regardless of mail contacts. 



TABLE 14 

Student Response Rates with Varying Amounts 
ai.d Kinds of Contacts 



Percent 


Al 1 


2 Mai 1 


3 Mail 




Returns 


Reports 


Contacts 


Contacts 


Telephone* 


l-IO 










1 1-20 










21-30 


6 


2 




1 


31-40 


13 


2 




2 


41-50 


4 


3 






51-60 


10 


4 




1 


61-70 


10 


1 


1 




71-80 


14 


5 


1 


2 


81-90 


8 


3 


1 


4 


91-100 


1 1 


4 


1 


4 


Totals 


76 




T 


74 



♦Regardless of mail contacts 



9. What is the general format of reports? For example, what Is 
their lengthy how many questions are asked, what balance exists between 
narrative and data, and how much unrelated material is included? 

Report length of community college follow-up studies of former 
occupational -technical students including appendices is typically 
20 pages or fewer (Table 15). 

The number of questions asked of students varies broadly. Of the 
51 reports which included sample questionnaires, 70 percent were found 
to contain 20 questions or fewer (Table 16). It should be noted that 
counting the number of questionnaire items is an imprecise technique 
due to differences in numbering systems. One college may obtain the 
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same amount of data In one three-part question as another collects In 
three separate questions. Informal adjustments were made to partially 
correct for this problem. 



TABLE 15 



Length of Reports 



Number of Pages 


N* 


% 


l-IO 


34 


39 


1 1-20 


16 


19 


21-30 


13 


15 


31-40 


7 


8 


41-50 


3 


4 


51-60 


5 


6 


61-70 


3 


4 


More than 70 


4 


5 


Tota 1 reports 




100 



*The mean number of 
pages was 24. 



TABLE 16 



Number of Items In Questionnaires 



Number of Questions 


N 


i 


l-IO 


19 


37 


il-20 


17 


33 


21-30 


7 


14 


31-40 


3 


6 


41-50 


2 


4 


51-60 


2 


4 


61-70 


J_ 


2 


Total reports with 






sample questionnaires 


51 


100 



The balance between data and narrative In research reports was coded 
in three categories appearing In Taole 17. Approximately one-half of the 
reports contained data In tables, graphs, charts and lists, with very 
little Interpretation or explanation of findings. Just under one-half 
of the 85 studies presented a mix of data and narrative. 
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TABLE 17 

Balance Between Narrative and Raw Data 



N 



Data predominantly 43 50 

Data and narrative 38 45 

Narrative predominantly _4 5 

Total reports 85 TO 



It was found that slightly less than one-half of the follow-up 
reports contained information on both transfer and occupational -technical 
students (Table 18). Only three reports of the 85 could be considered 
as unrelated documents used to convey follow-up data. This group 
contained a faculty bulletin, a placement newsletter, and a college 
newsletter. 



TABLE 18 

Follow-Up Studies Pertaining 
To Transfer As Well As To 
Occupational -Technical Students 



N i 

Reports on 0-T students 43 51 
Reports on both 0-T and 

transfer students 42 49 

Total IF 



10. Are statistical procedures employed^ such as tests of 
significance or of non-response bias? 

Only one of the follow-up study reports reviewed contained any 
Indication of the use of a statistical test or other Investigation of 
non-response bias (Table 19). Six reports employed some type of 
significance test either to determine whether samples-based findings 
were representative of a larger population or to compare results from 
two or more subgroups (Table 20). 

1 1 , What number of community colleges obtain employer evaluations 
of skills and work attitudes of former students? 

Only 6 of the 85 community college follow-up reports Included 
employer evaluations of former students (Table 21). 
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TABLE 19 



Follow-up Studies Reporting 
A Test of Non-Response Bias 





N 


% 


No test of non- response bias 
Employed a test of non-response 
bias 

Total reports reviewed 


Oil 

84 
1 


1 


TABLE 20 






Follow-up Studies Reporting 
Tests of Significance 






N 


1 


No tests of significance 
Employed tests of significance 
Total reports reviewed 


79 
6 
85 


93 
7 

100 


TABLE 21 






Follow-up Reports Containing Employer 
Evaluation of Former Students 




N 


i 


Contain employer evaluations 
No employer evaluations 
Total reports 


6 
79 


7 

93 
100 



12. Are data controlled for pre-enrol Iment experiences of former 
students? For example, do colleges report current salaries of former 
students without isolating students who had substantial prior experience 
or education in their occupations? 

Only six of the follow-up reports contained distinctions between 
novice and veteran workers in reporting such data as salaries and 
advancement (Table 22). However, none of the six made systematic 
efforts to separate findings for beginners and experienced workers. 
Rather, they made isolated notations on career longevity. 
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TABLE 22 



Reports Containing Distinctions 
Between New and 
Experienced Worl^ers 







N 


i 


Dist 


net ion made 


6 


7 


No dl 


stl notion made 


79 


93 


Tota 


reports 


85 


100 



13. Do colleges conduct longitudinal studies in order to measure 
career advancement or to obtain nore mature reflections of former students? 

Longitudinal studies are those which periodically contact former 
students during the years after college. Such studies might, for example, 
obtain information from the same students after I, 5 and 10 years. Only 
five longitudinal studies were Identified (Table 23). Twenty percent 
of the follow-up studies did compare responses of former students from 
different graduating or departing c I asses • Such studies are not truly 
longitudinal but are commonly termed comparative. 



TABLE 23 



Longitudinal ity of Fo I low-Up Studies 





N 


i 


Longitudinal 


5 


6 


Comparative 


17 


20 


One year or class only 


63 


74 


Total 


85 


100 



14. Given frequently low student questionnaire response rates In 
community college follow-up studies of former occupational-technical 
students, are questions being asked which could be eliminated bv obtaining 
data from college records? 

Nearly 70 percent of the 51 student questionnaires required students 
to supply data which might be obtained from college records (Table 24). 
Source high school, age, college major, credit hours earned and other 
registrars* data were not only requested of students but were also among 
the first questions presented in questionnaires. 
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Another category of Information, "enrollment period data" (Table 24), 
Is also often requested. For example, information on student financial 
aid and participation in student activities was requested by many follow- 
up researchers. Such data might be obtained by means other than student 
questionnaires. 



TABLE 24 



Questionnaires Obtaining 
Registrars* Data or Other 
Enrollment °eriod Data 





N» 




Registrars' data 


35 


69 


Enrollment period data 


16 


31 


Total questionnaires 


51 





*Addition to 51 and 100 percent is 
coincidental, since some questionnaires 
requested both registrars* data and 
enrollment period data. 



1 5. How many colleges request baccalaureate and post-baccalaureate 
transfer information from former occupational -technical students? 

16. How many colleges solicit educational evaluations and advice 
from former students ? 

Seventy-eight percent of 51 student questionnaires were found to 
contain Items which requested students to evaluate the community college 
they had attended (Table 25). A similar number of questionnaires solicited 
baccalaureate and post-baccalaureate information from students. The 
prevalence of baccalaureate-oriented questions would seem to Indicate 
concern for college transfer, even though occupational-technical programs 
are designed to provide preparation for employment. 
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TABLE 25 

Questionnaires Obtaining Student Transfer 
Information and Student Advice or Evaluations 





N 


% 


Request college transfer 






1 nformatlon 


41 


80 


Solicit student advice 






and evaluation 


40 


78 


Tota 1 questionna i res 5 1 







US" 

32 



17. To what extent are follow-up studies designed to measure 
attainment of educational goals? 

18. Which goals received the greatest amount of attention by 
community college follow-up studies of former occupational -technical 
students? 

The data collected or reported in 85 follow-up studies were compared 
with 44 educational goals for occupational-technical education. This 
content analysis procedure, described in Section II, revealed that the 
research emphasis of the group of follow-up reports was primarily on 
employment, income, and advancement of former students and on financial 
and manpower development of communities (Table 26). More than one-half 
of the studies collected or reported data useful in measuring the 
attainment of these five highly tangible, economic goals. 

Each of the other 39 goals in the inventory was important in the 
design of less than one-half of the reports reviewed. The goals of 
college satisfaction, program completion, and career satisfaction were 
important in t-he design of between 25 and 31 percent of the studies. 
All other goals were given attention by less than 20 percent of the 
reports. 

The authors recognize that few colleges have the resources to design 
and implement comprehensive follow-up studies which would obtain data 
relevant to all the educational objectives In an extensive 44-item 
inventory. However, there are several important goal areas which 
community colleges should not Ignore. For example, 13 or fewer of 
the 85 studies were concerned with students' awareness of their interests, 
abilities or career alternatives. Only eight studies investigated 
ethnic or racial differences in student outcomes. Just five studies 
attempted to Identify the college's influence on socio-economic 
mobi I i ty. 

Authorship of Reports 

Only 64 of the 85 follow-up reports showed the authors' identity 
(Table 27). More than three-fourths of these 64 reports were conducted 
in the office of institutional research (4l5t) or the placement office 
(365«). 
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TABLE Z6 

Follow-up Studies Giving Attention to Each of 44 Educational Goals 



Goals 


!i 


% 


Skills for employment 


82 


96 


Skills for adequate income 


60 


71 


Area manpower needs 


51 


60 


Economic development 


51 


60 


Skills for advancement 


50 


59 


Opportunity for student satisfaction 


26 


31 


Program completion 


24 


28 


Opportunity for career satisfaction 


21 


25 


Individual educational interest 


16 


19 


Eliminate financial barriers to enrollment 


14 


16 


Career i nformation 


13 


15 


Job seeki ng ski 1 1 s 


13 


15 


"Open door" opportunities for education 


12 


14 


Student* s knowledge of aptitudes 


12 


14 


Minimize college-caused attrition 


12 


14 


Qe-f-fer student understanding of interests 


1 1 


13 


Better student understanding of abilities 


10 


12 


Upgraded skills for experienced workers 


10 


12 


Effective student human relations skills 


9 


1 1 


Convenient time and place 


8 


9 


Equal opportunity for minorities 


8 


9 


Employee retraining needs 


7 


8 


Improved learning skills 


6 


7 


Effective work habits 


6 


7 


Positive attitudes toward work 


6 


7 


Information on working conditions 


5 


6 


General education competencies 


5 


6 


Socio-economic mobility 


5 


6 


Social skills and new friends 


4 


5 


Understanding of future needs 


3 


4 


Leadership skills 


3 


4 


Career entry ski 1 Is 




2 


Elimination of enrol 1 men*, barriers 


2 


2 


Career licensure or certification 


2 


2 


Society or union membership 


2 


2 


Leisure time fulfillment 


2 


2 


Civic pride 


2 


2 


Opportunities for the handicapped 


1 


1 


Awareness of change in careers 


1 


1 


Knowledge of democratic principles 


1 


1 


Ability to cope with personal problems 


0 


0 


Consumer education 


0 


0 


Law abiding citizenship 


0 


0 


Job safety 


0 


0 
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TABLE 27 

Title or Office of Fo I low-Up Report Authors 



N i 

Director or Office of Institutional Research 26 41 

Director or Office of Placement 23 36 

Division or Department 6 9 

Counselor 4 6 

Dean of Student^, or Office of Student Affairs 3 5 

Director of Instructional Services _2 3 

Total reports with authorship intllcated ' ■ '64 -108 
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SUMMARY 



This report reviewed follow-up studies of former occupational- 
technical s^udents at community colleges throughout the nation. The 
purpose of tne -oview was to evaluate the effectiveness of follow-up 
studies in measuring attainment of educational goals, as stated In 
community college catalogues and literature. This evaluation was made 
by studying research emphases, motivations, and procedures for studies 
conducted by community collage respondents. 



Procedure 



The study population consisted all 720 community colleges in 
the United States which began offer in^ occupational -technical education 
prior to September, 1970. Data were gathered in two separate procedures. 
First, a questionnaire was returned by 522 community colleges, along 
with copies of follow-up study reports. The questionnaire provided 
Information on research motivations and frequency. Second, 85 follow-up 
study reports were content coded by means of a content analysis coding 
form. A 44-ltem goc^ls Inventory for occupational -technica I education 
was prepared in order to measure the educational goal emphases of 
fol low-up studies. 



Findings 

Nearly all community colleges conducted some sort of follow-up 
study of former occupational-t^^hnlcal students. However, less than 
one-half conducted formal studies resulting In written reports. The 
range of quality among research reports was broad. Studies were 
conducted for a variety of reasons. Including curriculum evaluation, 
external accountability, and student services evaluation. Follow-up 
research was designed by college staff members with very little outside 
consultant advice or reference to texts and guides. Approximately 
60 prTcent of community colleges attempted to consult follow-up studies 
conducted by other institutions as they designed their own follow-up 
research. However, less than one-third of these were satisfied with 
the availability of other studies as a design source. 

Over three-quarters of the 85 foi low-up studios which were reviewed 
excluded non-graduates. In their follow-up research community colleges 
typically attempted to contact all former students of a given year or 
graduating class. Approximately one-fourth chose a sub-population or 
sample. 

Many follow-up studies of former occupationa I -technica I students 
contained inadequate information about procedures used. Based on the 
information available, around 60 percent of the community colleges 
appear to have made only one mall contact with students In ord^r to 
obtain data, and the remainder employed successive contacts. 
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Follow-up reports were typically less than 20 pages in length. The 
longer reports often included copious details such as lists of names and 
addresses of students, employers, etc. Of the student questionnaires 
reviewed. 70 percent contained 20 questions or fewer. Just over one-half 
of the reports consisted primarily of uninterpreted data. Such reports 
contained lists, tables and charts which the reader must interpret In his 
own fashion. Statistical analysis did not typically include tests of non- 
response bias or of the inferential value of samples-based findings. Less 
than 10 percent of the follow-up studies contained employer evaluations 
of student preparation or information on previous work experience. A 
similarly small percentage of reports was longitudinal in nature. 

Questionnaires were included with 51 of the 85 reports which were 
in the content analysis group.' Approximately 70 percent of the 
questionnaires contained items designed to obtain data which are often 
available In registrars* files, such as subject major and source high 
school. Questionnaire items on student transfer to other institutions 
and on student opinion of courses, counseling, and facilities were very 
common and appeared In approximately 80 percent of the questionnaires. 

Each report was reviewed in detail to determine whether it contained 
useful data for measuring goal -attainment, based on the 44- item goals 
inventory for occupational-technical education. Attainment was measured 
for only five goals in one-third or more of the reports. These goals 
concerned employment, income, career advancement, manpower development 
and economic growth. The attainment of such goals as equality of 
opportunity, positive attitude toward employment, and socio-economic 
mobility was evaluated by fewer than 10 percent of the institutions. 
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RECOMMENDATIONS 



The authors are aware that conmunlty colleges are constrained by 
limited resources for institutional research. However, follow-up 
research has great importance in building confidence and public support 
for the institution. Improved research on educational outcomes may 
well provide increased firdncial support for community colleges. 



Recommendations for Fol low-Up Study Improvement 

Ba§ed on the pi-eceoing finding^ th^ authors, have developed a list 
cf twelve recommendations for Improving community college follow-up 
studies of former occupational-technical students: 

1 . Seek advice from outside consultants who are proficient in 
fol low-up techniques . Only 14 percent of 391 responding 
colleges had sought the advice of research consultants. 

2. Utilize published texts on planning, conducting, and reporting 
studies . Only 17 percent of 391 responding colleges had used 
fol low-up guides. 

3 . Include non-graduates In research populations or samples . 
Less than one-fourth of the 85 reviewed reports Included 
findings about non-graduates. Since many students in 
occupational -technical programs at community colleges do 
not i^aduate, the post-college activities and perceptions 
of these former students should be sought. 

4. Employ representative sampling techniques . Only two of the 
85 follow-up reports indicated that random or systemlzed 
sampling procedures had been used. These sampling procedures 
can substantially reduce research costs at large institutions. 

5. Set an acceptable student response rate and implement means 
to achieve It . Thirty-three of the 85 colleges in the review 
group reported response rates of 70 percent or more, 
demonstrating that these rates can be achieved. Determining 
methoos of reaching a desirable response rate is an Important 
part of research planning. 

6. Limit student questionnaires to Information not obtainable 
from college records . Nearly 70 percent cf 5L questionnaires 
requested such information as college major or year of 
graduation. Many asked for financial aid and other information 
which is easily obtained from college records. Such questions 
unnecessarily lengthen questionnaires and might have adverse 
effects on response rates. 

7. Test for non- response bias . Approximately half of the studies 
reported response rates of less than 50 percent. Yet only one 
report Indicated the use of a test of non-response bias. 
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8. Distinguish between experienced workers and career-entry workers . 
More than half of the 85 follow-up reports contained information 
on the salaries and career advancement of former students, but 
only six made any distinction between veteran and novice employees. 
The effects of occupational-technical training on employment 
opportunities cannot be adequately judged without such refinements 
in data. 

9. Conduct longitudinal studies to measure career advancement and 
to obtain experience-based information . Three- fourths of the 
research studies pertained only to recent students (6 to 18 
months after college). Responses are also needed from former 
students who ha v^ been employed for three, five, and ten- years - 
after leaving the community college. 

10. Obtain employer evaluations of former occupational -technical 
students . Only six studies reported collecting or using data 
from employers. The opinions and suggestions of employers are 
valuable in assessing educational programs intended to build 
job ski I Is. 

1 1 . Provide interpretive material and descriptions of populations 
and procedures . Over half of the reviewed reports consisted 
primari ly of raw data. Several reports did not specify whether 
the subjects were graduates or non-graduates. One of eight 
contained no response rate information or data from which 
response rates could be calculated. Two of three reports did 
not indicate the number of student contacts, and one of four 
did not specify authorship. 

12. Relate follow-up study results and interpretations to educational 
goa I s . A survey of educational goals in community college 
catalogues and literature yielded 44 goals, all but a few of 
which were ignored in most of the reviewed reports. Only 21 
studies measured career satisfaction, and 13 reported on the 
effectiveness of career information. Nine studies provided 
information on former students* human relations skills, five 
were concerned with socio-economic mobility, and only two 
reported on employment licensure and union or occupational 
association membership for former students. Community college 
researchers should relate their results especially to goals 
listed in their own catalogues and master plans. 
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Suggestions for Additional Research 



This study was limited to occupational-technical education at 
community colleges* Substantial numbers of occu pat iona I -technical 
students receive their training at technical institutes and private 
two-year colleges. Follow-up studies conducted at these Institutions 
should be reviewed for the purpose of discovering model practices. 

This research has revealed large differences In student response 
rates from one follow-up study to another. A study should be conducted 
to examine factors which influence student cooperation In follow-up 
research. _ . . 

Finally, there appears to be great need for a comprehensive guide 
to planning, conducting and reporting follow-up studies of former 
occupational -technical students. 
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APPENDIX A 



TWO-YEAR COLLEGE CATALOGUES FROM WHICH 
EDUCATIONAL GOALS WERE DERIVED 



Col 



lege 



Anchorage Community College 
Anne Arundel Community College 
Bellevlew Community College 
Brevard Community College 
Cape Cod Community College 

Cerritos Col lege 
Clayton Junior College 
Columbia Basin College 
Cooke County Junior College 
Daytona Beach Community College 

East Central Junior College 
Essex County College 
Frederick Community College 
Grossmont Col lege 
Gulf Coast Junior College 

Highland Park Col lege 
Iowa Central Community College 
Johnsun County Community College 
Kingsborough Community College 
Lakewood State Junior College 

Los Angeles Harbour College 
Macomb County Community College 
Massachusetts Bay Community College 
Metropolitan Junior College District 
Mount Hood Community College 

New Mexico Junior Col lege 

North Hennepin State Junior College 

rwal k Community Col lege 
Olympla Col lege 
Palm Beach Junior College 

Phillips County Community College 
Rhode loland Junior Col lege 
St. Johns River Junior Collage 
San Jose City College 
Sinclair Community College 

South Georgia College 
Tarrant County Junior College 
Trinidad State Junior College 
University of Kentucky Community 

Col lege System 
Ventura Col lege 

Wayne Community College 
Westchester Community CoMege 



State 

Alaska 
Maryland 
Washington (St) 
Florida 
Massachusetts 

Cal Ifornia 
Georgia 

Washington (St) 

Texas 

Florida 

Mississippi 
New Jersey 
Maryland 
Cal Ifornia 
Mississippi 

Michigan 
Iowa 
Kansas 
New York 
Minnesota 

Cal ifornia 

Michigan 

Massachusetts 

Missouri 

Oregon 

New Mexico 
Minnesota 
Connecticut 
Washington (St) 
Florida 

Arkansas 
Rhode Island 
Florida 
Cal ifornia 
Ohio . 

Georgia 
Texas 
Colorado 
Kentucky 

Cal ifornia 

North Carol Ina 
New York 
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APPENDIX B 



COVERING LETTER AND QUESTIONNAIRE 
SENT TO COMMUNITY COLLEGES 



VMOINIA DBPAflTMCNT COMMUNITV COLLSMS 



February 7, 1973 



Dear President: 



We are conducting a natlon^-wlde survey of fol tow-up studies of former 
career or occupational -technical students which have been conducted since 
January, 1967, and which describe or analyze educational outcomes of post* 
college activities o' students. The Virginia Department of Community Colleges 
Is conducting this study in consultation with a team of experts in post- 
secondary career and occupational-technical education, and with the encour- 
agement of the AACJC. 

The purpose of our survey Is to overcome t}.e Isolation of follow-up 
studies of former career or occupational -technical students, by publishing 
a national review of the frequency, emphases, procedures and findings of 
such studies. Relatively few local ly-proauced follow-up studies are 
disseminated through the ERIC Clearinghouse for Junior College Information. 

We make two requests: 

I* That you comolete the brief questionnaire which follows. 

2* That you mail single copies of follow-up studies conducted at 
your Institution which meet the description above. 

A postage-paid envelope Is enclosed for your convenience In returning 
the questionnaire. If you have any. questions about this request, please 
telephone Mr. William G. Williams, Research Associate for this project at 
703-770-4272. 



We greatly appreciate your help. 



Very truly yours. 




0 1 rector 

FAS:Ggw Research and Planning 

Enclosure 



•11 
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RESEARCH SURVEY QUESTIONNAIRE 



Rtgardless of whether you are sending copies of followup reports, m 
MUld like you to complete and return the following questionnaire. 

INSTRUCTIONS: Place the appropriate number In the blank provided to 
the left of each Item, except where noted. 



I. Type of control of your Institution. 
I Public 

7 Indepondeot church- re I a tod 
T ' Independent nondenom I national 
J Proprietary 



2. Type of student body. 
I Coeducational 

7 Predominantly men (90$ or more) 
J Predominantly women {90% or more) 

3. Show the approximate fallf 1972, headcount enrollment at yoar 
col lege. 

a . In career or occupational-technical programs 

b . In transfer programs 

(Career or occupational-technical programs Include those which 
lead to an associate degree, certificate, or diploma for occupa- 
tional preparation In trade, techn.lcal, or paraprofesslonal 
employment. Do not Include students who are in foundations or 
developmental programs, or those who are not In any specific 
programs.) 

4. Please Indicate the areas In which your college offers occupational- 
technical programs. Mark (X) each which applies. 



a. Agricultural 

1>. Business and Secretarial 

j:. Engineering and Related Technologies 

jd. Health Careers 

ja. Publ Ic Ser-vico 

"f. Other, plea:>e 1 1st 



5. Please Indicate the status of your follow-up studies of former 
career or occupational-technical students. 



I One copy of each appropriate study Is being sent 
? liavo reports, but do not wish to circulate them 
5 Have only Informal studios for which ihcre are no written reports 
7 Havo not conducted follow;up studies of career or occupatlonal- 
technical students. 
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6. If you arc sending one or more reports, do you wish your Institution 
to remain anonymous? 



I Yes 
? No 



7. How Important were each of the considerations listed below In 
motivating your follow-up research? If you have conducted 
several follow-up studies for different reasons, try to Indicate 
your most common or lyplcal priorities. Mark each as (I) very 
Important, (2) some Importance, or (3) little or no Importance. 

jB. Accountability to the public 

"b. Personnel evaluations 

"c. Requests from legislature, trustees, accrediting agencies, etc. 

jl.' Curriculum evaluation 

jB. Student services evaluation 

"*f. Other, please list 



6. 




How Important were each of the following In shaping the design 
of your follow-up research? If you have conducted several 
follow-up studies and they have had design input from different 
sources, tr/ to indicate the most common or most typical source 
of design Ideas. Marl^ each as (1) very important, (2) some 
Importance, or (3) littl-e or no Importance. 

Faculty members 

Administrators 

College's research staff 

Outside consultant 

Other follow-up studies 

Published guides such as O'Connors* 

Other, please 1 1s t 



9. Are you satisfied with the availability of follow-up reports 
prepared at other colleges for the purpose of comparing your 
procedures and findings? 

I Yes, satisfied 
I No, not satlsfl . 

7 Not applicable, have not attempted to consult other reports 



*Thomas J. O'Connor. Pol low-Up Studies In Junior Colleges: A Tool for Institutional 
ImproveiKsnt. Washington: American Association of Junior Colleges, 1965. 
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JO. Does your Institution regular ly conduct exit Interviews of with- 
drawing students In order to determine why they are discontinuing 
attendance at your Institution? 

I Yes 
7 No 



J I. Has your Institution conducted one or more surveys of the non-student 
population In your service area since January, 1967, In order to 
obtain such Information as why persons do not attend the college, or 
to make demographic comparisons between students and non-students? 

I Yes 
1 No 



12* If your former students have been included In state-wide or 
regional follow-up studies since January, 1967, please famish 
the following Information: 

Name of the 

Investigating agency_^ 



Address 



Title of the report_ 
Report author 



MANY THANKS FOR YOUR VALUABLE HELP! 



Name of person completing this form Title Phono 
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REMINDER LETTER TO COMMUNITY COLLEGES 

VMOINIA OSPAMTMSNT OP COMMlMTrV COU.« 



March 5, 1973 



Dear President: 

Several weeks ago we mailed you a request for Information related 
to a nation-wide survey of studies of career or occupational -technical 
students at two-year colleges. Specif leal ly, we requested copies of 
foi low-up studies completed at your college and the completion of a 
short questionnaire. 

If you have answered our request, please Ignore this remlndor. 

Since we have not received your reply to date, we are sending a 
duplication of the earlier correspondence. Our review of follow-up 
study procedures and findings shculd result In the advancement of 
follow-up research design and outcomes evaluation at two-year colleges. 
Kowever, a high rate of response to our request Is necessary If the 
survey Is to be successful. 

Thanks for your help. 



Very truly yours. 




Research and Planning 

FAS:egw 
Enclosure 
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APPENDIX D 



CONTENT CODING FORM FOR 
FOLLOW-UP STUDY REPORTS 



NATIONAL FOLLOW-UP STUDY WTOUT CODING SllDET PAKT 1 - GENERAL CllAltACTERISTICS 

Institution name . Code _ 

Address Class 



_ 1. (1) all grads (2) sample grade (3) all non (4) sample now (3) all stu 

(6) sample stu 

_ 2. (1) agriculture (2) business and sec '(3) engineering and tech. 

W health (5) pub. svs. (6) other 

_ 3. Sample method: (1) random (2) system (3) arbitrary 

^ 4. contacts: 1 2 3 4 5 or more 4.b telephone (1) yes (2) no 

5. response rate, grads: (1) less than 40% (2) 40/49 (3) 50/39 (4) 60/69 

(5) 70/79 (6) 80/89 (7) 90/100 

6. response rate, non: (1) less than 40% (2) 40/49 (3) 50/59 (4) 60/69 

(5) 70/79 (6) 80/89 (7) 90/100 

^ 7* number of report pages 

8. number of questions 

_12« context: (1) independent (2) related study (3) non-related study 

^13* (1) data predominantly (2) data and narrative (3) narrative predominantly 

^14* check non*response bias: (1) yes (2) none indicated 

JL5* test of significance: (1) yes (2) none indicated 

^16. reports from employers: (1) yes (2) none indicated 

^17. control for: (1) entry (2) update (3) retrain (4) none indicated 

^18. longitudinal f;tudy: (1) yes (2) part of one (3) no or no indication 
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PART 2 « COAL ITEMS 



Gene ral , 

19. Opportunity for education 

20. Time and place 

21. Cost and F.A. 
22* Manpower needs 

23. Economic Development 

Individual Differences 

24. Individual Interests 

25. Retrain 

26. Upgrade 

27. Career entry 

28. Minorities 

29. Handicapped 

30. Ed. disadvantage 

31. Enrollment barriers 

Sel f_ Unde rs t and Init 

32. Abilities 

33. Aptitude 
34« Interests 

35« Program placement/ completion 

36. Dropout reasons 

37. Future needs 

38. Career Information 

39. Working conditions 
40* Change 



Skills and Attitude for Work 

41« Skills leading to employment 

42* Advancement 

43. Work habits 

44. Work attitude 

45. Human relations 
46« License 

47. Society cr union 

48. Job seeking skills 
49« Job safely 

50. General education 

Reward tnR Life 

51. School satisfaction 

52. Soclo-eccnomlc mobility 

53. Adequate Income 

54* Occupational satisfaction 

55. Leisure time 

56. Social skills 

57. Personal problems 

58. Consumer education 

Citizenship 

59. Leadership 

60. Democratic principles 

61. Law violations 

62. Civic pride 



PART 3 * QUESTIONNAIRE ITEMS 



63. Registrar's data 

64. Enrollment period Information 

65. Student transfer 

66. Student advice and evaluation 
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APPENDIX E 



CONTENT CODING PRXEDURES FOR 
FOLLOW-UP STUDY REPORTS 



The contents of the 85 community college follow-up reports were 
coded in items I through 66 of the three-part coding form In Appendix D. 

Information coded in Part I (items I through 18) pertains to 
follow-up research procedures and report format. The coding system Is 
described below: 

Item I; Shows whether the research subjects were graduates only 
(options \ or 2), non-graduates only (options 3 or 4), 
or former students regardless of program completion 
(options 5 or 6). The first option In each of the three 
pairs (I, 3 or 5) was used to designate that an entire 
population rather than a sample or sub-population was used. 

Item 2. In cases where follow-up studies included subjects who 
were limited to one or two curricular areas, options 
one through six were used to identify the area. 

Item 3. When the information In Item I revealed the use of 

sampling, the method of sampling was indicated by means 
of options I , 2 or 3. 

Item 4a. This item shows the number of attempts which were made 
to contact former students by means of mailed contacts. 

4b. Any attempts to contact students by telephone were coded 
as I • 

Items 5 These items were found to be unsatisfactory and were not 
& 6 used. An alternative device of writing the exact aggregate 

percentage combined the response rates for all subjects 
regardless of program completion, curriculum, etc. 

Item 7. The number of report pages was coded and included all 
attachments or appendices. 

Item 8. The number of questions contained in each student question- 
naire was coded in those cases where the follow-up report 
contained a sample questionnaire. The validity of this 
code is not precise due to college-to-college Inconsisten- 
cies in numbering techniques. 

Items 9, These coding categories were deleted. 
10 & II 

Item 12. Independent reports (option I) are those which pertain to 
occupational -technical students exclusively. Option 2 was 
used to Indicate that occupational-technical follow-up 
Information was contained as part of some report larger in 
scope. Most typically "related studies" contain follow-up 
data on both 0-T and transfer students. Option 3 was used 
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to indicate the occurrence of occupational -technical data 
in a non-related document such as a college newspaper or 
annual report. 

Item 13. This item describes the balance between data and narrative. 

Items 14, These items have only two options each and merely show the 
15 4 16 presence or absence of the three procedures in the r3ports. 
Any test of non-response bias, a test of the population 
significance of samples-based data or any attempt to 
contact employers of former students are indicated by 
options I in each item. No attempt was made to judge 
whether the three procedures were used properly. 

Item 17. Options show whether the researchers distinguished between 
career entry students and those who had prior experience. 
Advancement and salary data, for example, are seriously 
biased when veteran workers are included with beginners. 

Item 18. A longitudinal study is one which contacts the same subject 
more than once over a period of time. Reports which 
included longitudinal comparisons were coded I. Optfon 2 
was used to identify a report which was a duplicate contact 
of a group of former students, but which did not report 
comparisons. Option 3 identifies those reports which are 
based solely on one follow-up contact of subjects. 

Items 19 through 62 on coding form A consi itute Part 2 of the 
coding form and are related to the educational goals inventory presented 
in Section III. Each of the numbered items represents consecutively the 
goals in the inventory. Brief, telegraphic key words were used to 
identify each goal for the coder. 

Each of the 85 follow-up reports was coded twice together with 
the accompanying student questionnaire, if present. Determinations 
were made as to whether the data which were presented and/or collected 
could be related to the measurement of the attainment of each of the 
educational goals. No attempt was made to delermine whether the college 
researchers had accurately measured the attainment of an educational 
goal, but rather whether they had demonstrated interest in the goal 
by collecting or reporting relevant data. 

The sample evaluation measures in Section III consist of examples 
of the kinds of data gathered or requested which would result in a 
positive coding response. 

For example, coding sheet item 52 is "socio-economic mobility". 
The evaluation measures for the corresponding goal (No. 34) are 
"Former student educational level, career field prestige and other 
SES factors compared with those of parents". 

If a college follow-up report collected such data and/or reported 
them, the report was given credit for measurement interest in goal 34. 
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On the coding sheet this was accomplished by placing a circle around 
number 52. 

The coding of measurement Interest In an educational goal was 
not limited to the appearance of evaluation measures which are found 
In the goals Inventory. For example, a college might have asked Its 
former students whether It was the student's opinion that he was better 
educated than his parents or whether he thought his career would result 
in greater success than that enjoyed by his parents. 

Part 3 of the fo! low-up report coding form contains four Items 
numbered 63-66 which were used to Identify miscellaneous characteristics 
of the 51 student questionnaires which were coded. College requests 
for registrar's data, enrollment period Inform^^tlon, student transfer 
information, or advice and evaluations from former students was Indicated 
by circling the appropriate Item number. 
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APPENDIX F 



COMMUNITY COLLEGES WHOSE FOLLOW-UP 
REPORTS WERE CONTENT CODED 



Col lege 


State 


Cochise County Community College 


Arizona 


Westark Junior College 


Arkansas 


Canada Ccl lege 


Cal Ifornia 


Coast Comn.unity College District 


Cal I fornia 


Los Angeles City College 


Cal 1 fornia 


Los Angeles Harbour College 


Cal i fornia 


Los Angeles Pierce College 


Cal i fornia 


Santa Barbara City College 


Cal Ifornia 


Santa Rosa Junior College 


Cal i fornia 


Community College of Denver, 


Col orado 


Auraria Campus 




Mattatuck Community College 


Connecticut 


Broward Community College 


Florida 


Daytona Beach Community College 


Florida 


Mi am I -Dade Junior College 


Florida 


Santa Fe Community College 


Florida 


Dekalb Col lege 


Georgia 


Kauai Community College 


Hawaii 


Kishwaukee Col lege 


1 1 1 1 no I s 


Moraine Vw^lley Community College 


1 1 1 I no i s 


Sauk Val ley Col lege 


1 1 1 i no i s 


Spoon River College 


1 1 1 I no I s 


Des Moines Area Community College, 


Iowa 


Central Campus 




Iowa Central Community College 


Iowa 


Southeastern Community College 


Iowa 


Charles County Community Co-lege 


Maryland 


Harford Community College 


Maryland 


Montgomery College 


Maryland 


Delta Col lege 


Michigan 


Macomb County Community College, 


Mich igan 


South Campus 




Southeastern Michigan College 


Mich igan 


Mississippi Delta Junior College 


Mississippi 


Florrisant Valley Community College 


Missouri 


State Fciir Community College 


Missouri 


Northeast Technical Community College 


Nebraska 


Mercer County Community College 


New Jersey 
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Co 1 1 ege 


State 


Broom Community College 


New York 


Columbia Green Community College 


New York 


Dutches Community College 


New York 


New York City Community College 


New York 


Niagara County Community College 


New York 


S.U.N.Y. Agricultural and Technical College 


New York 


Morrisvil le Campus 




North Dakota State School of Science 


North Dakota 


Lakeland Community College 


Ohio 


Mt. Hood Community College 


Oregon 


Community College of Philadelphia 


Pennsyl van! a 


Harrisburg Area Community College 


Pennsyl van la 


Lehigh County Community College 


Pennsyl vania 


Ammari ! lo Col lege 


Texas 


Bee County College 


Texas 


Tarrant County Community Col lege 


Texas 


Tidewater Community College 


Virginia 


Germanna Community College 


V i rg 1 n 1 a 


Blue Ridge Community College 


Virginia 


Lord Fairfax Community College 


Virginia 


Shore 1 ine Community Col lege 


WashI ngton 


Everett Community College 


Washington 


Madison Area Technical College 


Wisconsin 


Milwaukee Area Technical College 


Wisconsin 


(Twenty-seven institutions requested anonymity.) 
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APPFNDIX G 



PROJECT STAFF AND CONSULTANTS 
Consultants 



Dr. Lewis R. Fibel 
Dean of the Col lege 
Keystone Junior College 
La Plume, Pennsylvania 



Dr. Angelo C. Gill ie 
Associate Professor of 

Graduate Studies and Research 
Pennsylvania iitato University 
University Park, Pennsylvania 



Dr. J. WadeGilley, President 
J. Sargeant Reynolds Community 

Co I I ege 
Richmond, Virginia 



Dr. Aaron J . Mi I ler 
Associate Profej.^or 
Col lege of Education 
Ohio State University 
Columbus, Ohio 



Pr i nc i pa I I n vest ! gator 



Research Advisor 



Project Director 



Dr. Samuel D. Morgan 

Assistant Professor of Vocational 

and Technical Education 
Virginia Polytechnic Institute and 

State Uni>'")rsity 
Blacksburg, Virginia 



Dr. Cheryl A. Of cinch 
Coordinator of istitutional Research 
Catonsville Com...unity College 
Catonsville, Maryland 



Dr. Arden Pratt, Dean 

Vocat i ona I -Techn i ca 1 I nst i tute 
Southern 1 1 1 i no i s Un i vers i ty 
Carbondale, 1 1 1 inois 



Kenneth G. Skaggs 

Occupational -Technical Special ist 

American Association of Community 

and Junior Col leges 
Washington, D. C. 

Staff 



Dr. Wi I I iam G. Wi I I iams 
Research Associate 

Dr. Ted 0. Gusti lo 
Former Coordinator, Research and 
Evaluation 



Dr. Fred A. Snyder 
D i rector 

Division of Researcfi jna Planning 
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